On the Representation and Combination of Evidence in Instance-Based Learning

In instance-based learning the classification of a novel instance relies upon experience given in the form of similar instances whose labels are already known. Each of these instances can hence be seen as an individual piece of evidence. In this paper, we elaborate on issues concerning the representation and combination of such pieces of evidence. Particularly, we argue that the information provided by similar instances must not be considered as independent. We propose a new inference principle that derives an evidence function specifying the available evidence in favor of each potential label. This principle, which is built upon a probabilistic (random field) model, takes interdependencies between stored instances into account and suggests a generalization of weighted nearest neighbor estimation.